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Abstract 

The Bonn Distributed FX (DiFX) correlator is a software correlator operated jointly by the Max- 
Planck-Institut fiir Radioastronomie (MPIfR), the Institut fiir Geodasie und Geoinformation der Uni- 
versitat Bonn (IGG), and the Bundesamt fiir Kartographie und Geodasie (BKG) in Frankfurt. 


1. Introduction 

The Bonn correlator is hosted at the MPIfR 1 in Bonn, Germany. It is operated jointly by the 
MPIfR and the BKG 2 in cooperation with the IGG 3 . It is a major correlator for geodetic observa- 
tions and astronomical projects, for instance those involving pulsar gating, millimeter wavelengths, 
and astrometry. 

2. Present Correlator Capabilities 

The Distributed FX correlator 4 was developed at Swinburne University in Melbourne, Aus- 
tralia by Adam Deller (and other collaborators). It has been adapted to the VLB A operational 
environment by Walter Brisken and the NRAO staff, and it has been developed for a number of 
years by the worldwide DiFX developers group. DiFX in Bonn is installed and running on a High 
Performance Compute Cluster (HPC cluster). 

Features of the software correlator cluster are: 

• 60 nodes (eight compute cores each) 

• four TFlops in the Linpack benchmark test 

• 20 Gbps Infiniband interconnection 

• 11 RAIDs (about 480 TB storage capacity) 

• one control node for correlation ( fxmcmager ) 

• one computer ( frontend ) for executing parallelized jobs on the cluster, e.g., post-correlation 
applications 

• one control computer ( appliance ) for installing and monitoring the cluster 

• closed loop rack cooling 

The correlator cluster is connected via 20 Gbps Infiniband to 14 Mark 5 units 5 used for playing 
back the data. If more than 14 playback units are required, and in the case of e-VLBI, data are 
copied to the raid systems prior to correlation. All Mark 5 units can play back all types of Mark 5 
data (A/B/C). The disk-modules in the Mark 5 are controlled via NRAO’s mk5daemon program. 
The available functionality includes all necessary functions such as recording the directories of the 
modules, resetting and rebooting the units, and module conditioning. 

1 http : //www3 .mpif r-bonn.mpg . de/div/vlbicor/ index_e . html 
"http : //www .bkg.bund. de/ 

! http : //www . gib . uni-bonn . de/ 

4 DiFX: A Software Correlator for Very Long Baseline Interferometry using Multiprocessor Computing Environ- 
ments, 2007, PASP, 119, 318 

'http : //www .hay stack. mit . edu/tech/vlbi/mark5/ 
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A summary of the capabilities of the DiFX software correlator is presented in Table 1. 


Table 1. Correlator capabilities. 


Playback Units 


Number available 

14 Mark 5 (four Mark 5A, two Mark 5B, and eight Mark 5C) 

Playback speed 

1.5 Gbps 

Formats 

Mark 5 A, Mark 5B, and VDIF 

Sampling 

1 bit and 2 bits 

Fan-out (Mark 5A) 

1:1, 1:2, and 1:4 

No. channels 

< 16 USB and/or LSB 

Bandwidth /channel 

(2, 4, 6, 8, and 32) MHz 

Signal 

Single- and dual- frequency; all four Stokes parameters for circular 


and linear polarization 

Correlation 


Geometric model 

CALC 9 

Phase cal 

Phase-cal extraction of all tones in a sub-band simultaneously 

Pre-average time 

Milliseconds to seconds 

Spectral channels 

Maximum number of FFT tested 2 18 

Export 

FITS export. Interface to MkIV data format which enables the use of 


geodetic analysis software and the Haystack fringe fitting program. 

Pulsar 

Pulsar with incoherent dedipersion 


3. Staff 

The people in the Geodesy VLBI group 6 at the Bonn correlator are: 

Arno Miiskens - group leader and scheduling of T2, OHIG, EURO, and INT3 sessions. 

Simone Bernhart - e-transfer supervision and operations, experiment setup and evaluation of 
correlated data, and media shipping. 

Alessandra Bertarini - experiment setup and evaluation of correlated data for both astronomy 
and geodesy, digital baseband converter (DBBC) testing, APEX fringe testing, Friend of the cor- 
relator. 

Laura La Porta - experiment setup and evaluation of correlated data, DBBC testing, and pro- 
gramming for automated preparation of correlation reports. 

The people in the astronomical group at the Bonn correlator are: 

Walter Alef - head of the VLBI technical department, computer systems and cluster administra- 
tion. 

Alan Roy - deputy group leader, support scientist (water vapor radiometer, technical assistance, 
development of FPGA firmware for linear to circular polarization conversion, and project manager 
for equipping APEX for millimeter VLBI). 

Helge Rottmann - software engineer for correlator development and operation, cluster adminis- 
tration, DBBC and RDBE control software, and Field System. 

6 http : / / www3 . mpif r-bonn . mpg . de/di v/ vlbicor/ geodesy/ index . html 
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Heinz Fuchs - correlator operator, responsible for the correlator operator schedule, daily opera- 
tions, and media shipping. 

Hermann Sturm - correlator operator, correlator support software, media shipping, and Web 
page development. 

Rolf Martens - technician maintaining cluster hardware and Mark 5 playbacks. 

Michael Wunderlich - engineer, development, and testing of DBBC components. 

Jan Wagner - PhD student, support scientist for APEX, DBBC development, and DiFX devel- 
oper. 

Armin Felke - FPGA programming for DBBC. 

Gino Tuccari - guest scientist from INAF, DBBC development, and DBBC project leader. 
David Graham - consultant (technical development, DBBC development, and testing). 

4. Status 

Experiments: In 2012, the Bonn group correlated 51 Rl, six EURO, five T2, five OHIG, 43 
INT3, and 16 astronomical experiments. 

e-VLBI: On average > 60% of the stations do e-transfer, and the number still increases. E.g., 
in the T2086 session, 16 stations participated in the observations, and 11 of them sent their data 
via high-speed network connection. The average amount of e-transferred data per week ranges 
from 4 to 6 TB considering only the regular INT3 and Rl experiments. Most transfers are done 
using the UDP-based Tsunami protocol. 

The total disk space available for e-VLBI data storage at the correlator is currently about 
125 TB. The Web page that shows current active e-transfers and helps to coordinate transfer times 
and rates on a first come- first served basis 7 , has been extended by information about the storage 
capacity at the three correlators in Washington, Haystack, and Bonn. The envisaged upgrade of 
the existing 1 Gbps Internet connection in order to meet the requirements of the higher observing 
rate foreseen within VLBI2010 has not yet been realized due to still existing issues concering 
funding. However, the transfer Web page currently seems well adopted by the community and 
even though meanwhile more than 60% of the stations nowadays transfer their observational data 
via Internet, we have merely been facing minor problems - if any - due to bandwidth limitations. 

DiFX software correlator: A graphical user interface was installed on the DiFX control 
computer, which simplifies the use of the software correlator. 

A branch version of the DiFX software correlator for RFI mitigation has been developed as 
part of a PhD project. DiFX RFI suppression via filtering fast off-source fringe rates is complete 8 . 
While it reduces ringing by RFI along UV plane tracks, it has proven ineffective at removing a 
residual constant RFI power in the affected channels, and it is not as powerful as the method for 
Focal Plane Array RFI excision also included in the DiFX library. 

DBBC: The Bonn group is involved in the development of the DBBC for the European VLBI 
Network (EVN) and geodesy. The DBBC is designed as a full replacement for the existing analog 
BBCs. The following stations have already bought one or more DBBCs: APEX, AuScope (Aus- 
tralia), Effelsberg, Onsala, Pico Veleta, Yebes, Wettzell, and Warkworth. Most of those DBBCs 
are already in regular use. High data rate modes with 2 and 4 Gbps are being tested. First test ob- 
servations at 2 Gbps and 4 Gbps were performed in June 2012 (at Yebes, Effelsberg, and Onsala). 

' http : //www3 .mpif r-bonn.mpg . de/cgi-bin/showtransf ers . cgi 

8 http : //www . radionet-eu. org/fp7wiki/lib/exe/f etch.php?media=jra: albius : rf ifringef ilter_pub . pdf 
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Regular testing with geodetic observations is performed for DBBCs at Onsala and Wettzell. 

In the summer of 2012, a project to develop the next generation DBBC called DBBC3 was 
started. In the first stage, a system which can handle 4 GHz bandwidth will be developed (DBBC3- 
L). In the second stage, the DBBC3-H will be able to sample the full frequency range of 1 to 14 Ghz 
without any downconversion required. 

APEX: The Bonn VLBI group has equipped the APEX telescope for VLBI observations at 
1 mm. The first successful fringe test took place in May 2012 on 3C 279 at 229 GHz with SMA 
(Hawaii) and SMTO (Arizona). The fringe spacing achieved was 29 microarcseconds, adequate to 
resolve the expected diameter of the shadow of the event horizon of 47 microarcseconds in Sgr A*. 
A fringe plot of these observations is shown in Figure 1. 
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Figure 1. Successful 1 mm fringe tests performed in March 2012, baseline APEX- Submillimeter Telescope 
Observatory (SMTO, Arizona), on 3C 279. The fringe spacing was 29 microarcseconds, the finest yet 
achieved. 


5. Outlook for 2013 

DiFX Correlator: The planning to replace the now five-year-old cluster with a more modern 
system will begin. 

e-VLBI: e-transfer tests with other antennas are planned or ongoing. 

DBBC: DBBC testing in the EVN stations that recently acquired DBBCs will continue. Wide 
bandwidth modes are also under test. Development of the DBBC3 will take place. 

APEX First real observations will take place at APEX in March 2013. 

Phasing up ALMA The group is involved in an international project to add array phasing 
capability to ALMA. This will enable its use as an extremely sensitive station in 1 mm VLBI 
experiments. 
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